RHOMBUS TILINGS OF A HEXAGON WITH THREE FIXED BORDER TILES 



THERESIA EISENKOLBLi 



Abstract. We compute the number of rhombus tihngs of a hexagon with sides a + 2, 6 + 2, c + 2,a + 
2, 6+2, c + 2 with three fixed tiles touching the border. The particular case a = b = c solves a problem 
posed by Propp. Our result can also be viewed as the enumeration of plane partitions having a + 2 
rows and 6 + 2 columns, with largest entry < c + 2, with a given number of entries c + 2 in the first 
row, a given number of entries in the last column and a given bottom-left entry. 



1. Introduction 

The interest in rhombus tihngs has emerged from the enumeration of plane partitions in a given 
box (which was first carried out by MacMahon [^). The connection comes from representing each 
entry by a stack of cubes and projecting the picture to the plane. Then the box becomes a hexagon, 
where opposite sides are equal, and the cubes turn into a rhombus tiling of the hexagon where the 
rhombi consist of two equilateral triangles (cf. m). The number of plane partitions contained in an 
a X b X c-box was first computed by MacMahon and equals 

mti' kiuitT' kimtr' kl 

In 0], Propp proposed several problems regarding "incomplete" hexagons, i.e., hexagons, where 
certain triangles are missing. In particular. Problem 3 of |Q asks for a formula for the number of 
rhombus tilings of a hexagon with sides 2n, 2n + 3, 2n, 2n + 3, 2n, 2n + 3 and angles 120°, where the 
middle triangle is missing on each of the longer borders. This turns out to be a special case of the 
following result (see Corollary ||) : 

Theorem 1. Let a, b, c be nonnegative integers. The number of rhombus tilings of a hexagon with sides 
a + 2,c + 2,b + 2,a + 2,c + 2,b + 2, with fixed tiles in positions r, s, t touching the borders a + 2,b + 2, c+2 
respectively (see Figure |i|o for the exact meaning of the parameters r,s,t) equals 

(r + 1),(. + lUt + lUc + 3 - tUa + 3 - r^b + 3 - ■s). ^^'--",^^ n^^^^^^^^ 

nfe=o 11^=0 11^=0 kl 

X ((a + l){b+ l)(c + l)(a + 2 - r)(6 + 2 - s)(c + 2-t) + {a+ 1)(6 + l)(c + l)rst 

- (a + 2 - r){b + 2- s)(c + 2- t)rst + (a + l)(c + 1)(6 + 2- s)[c + 2- t)rs 

+ {b+l){a+ l)(c + 2 - t)(a + 2 - r)st + (c + \){b + l)(a + 2 - r){b + 2- s)tr^ , 

where {a)n — a(a + 1) • • ■ (a + n — 1). 

As shown in Figure |^, the fixed tiles determine the tiling along the borders they touch. Thus, we 
can remove three strips of triangles and end up with a hexagon with sidelengths a, c + 3,6, a-|-3,c, 6 + 3 
and missing border triangles in positions r, s,t (see Figure The special case a = b = c = 2n and 
r^s = t^n+l solves Problem 3 of Q . This is stated in the following corollary. 
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a. A hexagon with fixed tiles on three borders. 



b. A rhombus tiling of the hexagon corres- 
ponding to the plane partition from (H). 



Figure 1. 




A hexagon with fixed tiles on three borders. 
The tiling of the dotted area is determined by the fixed tiles. 

Figure 2. 



Corollary 2. The number of rhombus tilings of a hexagon with sides 2n, 2n + 3, 2n, 2n + 3, 2n, 2n + 3, 
where the middle triangle is missing on each of the longer borders, equals 



((n + 2)2„) 



fc! 



-(n+l)3(3n + l)(3n + 2)2. 



(1) 



Theorem ^ has also an interpretation in terms of plane partitions. However, it makes only sense, 
if we view the plane partitions as planar arrays of nonnegative integers with nonincreasing rows and 
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A hexagon with sides a, c + 3,b, a + 3, c,b + 3, with triangles 
in positions r, s, t missing, where a — 2, b — 1, c = 1, r — 2, s = 2, t = 1. 



Figure 3. 



columns. For example, the plane partition of Figure |^b is represented by the array 

3 2 2 

2 2 ^''^ 
2 1 0. 

It is easy to see that a plane partition contained in an (a + 2) x (6 + 2) x (c + 2)-box is represented 
by an array of integers < c + 2 having a + 2 rows and 6 + 2 columns. Furthermore, the fixed tiles of 
Theorem |^ correspond to the conditions that b + 2 — s entries in the first row of the plane partition 
are equal to c + 2, r entries in the last column are equal to and the bottom-left entry is c + 2 — t (cf. 
Figure]^). So Theorem]^ has the following corollary. 

Corollary 3. The number of plane partitions contained in an (a + 2) x (6 + 2) x (c + 2)-box having 
6 + 2 — s entries equal to c + 2 in the first row, r entries equal to in the last column and c + 2 — t as 
the bottom-left entry equals 

(r + + lUt + lUc + 3 - tUa + 3-rUb + 3- ^)a ^^^-CUy '^iit?, , , , 

X ((a + + l)(c + l)(a + 2 - r){b + 2 - s){c + 2 - t) + {a + l){b + l)(c + l)rst 

-{a + 2- r){b + 2 - s)(c + 2 - t)rst + (a + l)(c + 1)(6 + 2 - s)(c + 2 - t)rs 

+ {b+l){a+ l)(c + 2-t){a + 2- r)st + (c + + l)(a + 2 - r){b + 2- s)ir) . 

For the proof of Theorem |l|, which we provide in Section 2, we proceed as follows. First, we use the 
fact mentioned immediately after Theorem 0, that it suffices to enumerate the rhombus tilings of a 
hexagon with sides a, c + 3, 6, a + 3, c, 6 + 3 and missing border triangles in positions r, s, t as shown in 
Figure 3. This can be expressed as a determinant by using the main theorem of nonintersecting lattice 
paths [2, Cor. 2] (see also Theorem 1.2]). The determinant is then evaluated by induction, using 
a determinant lemma from (see Lemma ^ and equation (^), a determinant formula published by 
Jacobi in 1841 (see |3j) but first proved in 1819 by P. Desnanot according to The formula is also 
closely related to C. L. Dodgson's condensation method. 
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2. Proof of Theorem [T| 

By the paragraph following Theorem ^ it is enough to show that the theorem holds for the number 
of rhombus tilings of a hexagon with sides a, c + 3, 6, a + 3, c, 5 + 3 and missing border triangles in 
positions r,s,t (see Figure 

We start the proof by setting up a correspondence between these rhombus tilings and certain families 
of nonintersecting lattice paths, where nonintersecting means that no two paths have a common vertex. 
The reader should consult Figure ^ while reading the following passage. Given a rhombus tiling of the 
region described above, the lattice paths start on the centers of upper left diagonal edges (the edges on 
the side of length a) and the two extra edges parallel to it on the two neighbouring sides. They end on 
the lower right edges (the edges on the side of length a + 3). The paths are generated connecting the 
center of the respective edge with the center of the edge lying opposite in the rhombus. This process 
is iterated using the new edge and the second rhombus it bounds and terminates on the lower right 
boundary edges. It is clear that paths starting at different points have no common vertices and that 
an arbitrary family of nonintersecting paths from the set of the upper left edges to the set of the lower 
right edges lies completely inside the hexagon and can be converted back to a tiling (see Figure ^). 

Then we transform the picture to orthogonal paths with positive horizontal and negative ver- 
tical steps of unit length (see Figure ^c,d). Let the starting points of the paths be denoted by 
Pq, Pi, . . . , Pa+i and the endpoints by Qq, Qi, . . . , Qa+i- Now we can easily write down the coor- 
dinates of the starting points and the endpoints: 

Fo = (0,c + 2-t), 

P, = {i-l,c + 2 + i), for i = 1, . . . ,a, 

Pa+i = (a + 6 + 2 - s, a + c + 2), 

Qj ^ib + j + X{j >r),j+ xii >r)), for j 0, . . . , a 1. 

Here, the symbol xH ^ ^) equals 1 for j > r and else. It ensures that the missing edge on the side 
of length a -f 3 is skipped. 

Next we apply the main result for nonintersecting lattice paths Cor. 2] (see also Theorem 1.2]). 
This theorem says that the number of families of nonintersecting lattice paths with path i leading from 
Pi to Qi is the determinant of the matrix with (i, j)-entry the number of lattice paths leading from Pi 
to Qj, provided that every two paths Pi —^ Qj, Pk — > Qi have a common vertex, if i < j and k > I. It 
is easily checked that our sets of starting and endpoints meet the required conditions. 

Since the number of lattice paths with positive horizontal and negative vertical steps from (a, h) 
to (c, d) equals ' ''^'^ number of families of nonintersecting lattice paths (equiv- 

alently, the number of rhombus tilings of our hexagon) by evaluating the determinant of the matrix 
M{a,b,c,r,s,t) = M = (Af [i, j])^+io> ^here 



' / b+c-t+2 

\c + 2~t~j~x{j> r) 

,.r- 1 f b + c + 3 \ . ^ 

M[i,j] = <. [,^.^ ... X .^-1 1 = 1,..., a, 
\b + J+ xi,]>r)-i + lJ 

f c + s 

- \3 +X{j >r) -a-2 + s 

We will do this using a determinant formula due to Desnanot (see 1^). Given a matrix A = 

{A[i,j])2j^Q, this formula states that 

(det A)(det A°;^;) = {det A°){det A^) - (det AJf)(det A^), (4) 

where denotes the matrix A with row i and column j deleted, and Aq'"^ denotes the matrix A with 
rows and n and columns and n deleted. (In general, given sequences of nonnegative integers U 
and V, we will use the symbol Ay to denote the matrix A with all row indices from U and all column 
indices from V deleted.) 
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1. 



a. A rhombus tiling of the hexagon of Figure 



b. The corresponding family of paths. 



<D 



Pi<> 



c. The path family without the rhombi. 



d. The orthogonal version of the path family. 



Figure 4. 



If we use (H) for A = M and n = a + 1, we get 

(detM)(detM°;+ii) = (det MO)(det M„"+i) - (det Mo"+i)(det M°+i). 



(5) 



In order to use (||) for the computation of det M, we need to know the determinants of Mq '^^^ , A/q , 
M^^l, Mo+^ and M°+i. We start with the evaluation of detMg '^+j^. We will employ the following 
determinant lemma from Lemma 2.2]: 

Lemma 4. 



det {{X, + A„){X, + An-i) ■ ■ ■ {Xj + + Bi){Xj + . . . [X, + B2)) = 

= n {x,~x,) n 

l<i<j<n 2<i<j<n 
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Lemma 5. Let 1 < r < a + 1. Then 

^ , ,^0 a+i iib + c + Syna + c + 2-r)l{b + r)! ni<.<,<a+i U - UVo {b + c + k + 4)«-i-^ 

det Mn ' 7i = , T , ^ . 

' ^ (a + 1 - r)!(r - 1)! n^ti' ((6 + + c + 2 - j)!) 

Proof. We start by pulling out appropriate factors from columns and rows and get 

0,a+l_ ff b + C + 3 



detM^'":!' = det , , , 



(_l)i:Li(a-.)-Q, {b + + 3)1 



{b + J+ Xij > r))Kc -3- Xii > r) + 2 + a)! 
X det {{b + j + xij>r)-i + 2),^l{j + xi3>r)'C-a~2)a-^). 

l<i.j<a 

Applying Lemma ^ with Xj = b + j + xij ^ ^fc = — c — fc — 2, = — fc + 2 and simplifying 
yields the desired result. □ 

The cases r = and r > a + 2 can be reduced to the previous lemma by observing that r occurs 
only in terms of the form x{j ^ Since xij ^ 0) — xij ^ 1) for j > 1 and x{j a + 1) = xiJ ^ ^) 
for r>a + 2, j<a we have 

detMo%i(a,5,c,0,5,t) = det M°;:+l{a,b,c,l, s,t) (6) 
det M°'^+j^ (a, 6, c,r,s,i) = det M°;^+j^(a, 6, c, a + 1, s, t) for r > a + 2. (7) 

Now we express all remaining determinants in equation (^) in terms of the determinant of Mq . 
Whenever a matrix does not depend on some parameter because of a deleted row, we will use a star 
in place of the parameter. It is easily checked that by appropriate relabelling of rows and columns 

det M°+;^ (a, 6, c,r,s,*) = det M^(a, 5- l,c+ l,r + l,s - 1,*), (8) 
det Af^+]^(a, 5, c, r, *, t) = det M^{a, c,b,a + 2 - r, c + 2 - t, *), (9) 
det Afo''+^(a,5,c,r, = det Mo°(a, c - 1, 6 + 1, a + 3 - r,c + 1 - t,*). (10) 

The remaining task is to evaluate detMg. We state the result for detMg in the following lemma. 
Lemma 6. Let 1 < r < a + 2. Then 

det M« = TT fiA+l+3f±l^\ (■'^ + l)c n.Ll^' 

° (c + i + 2)1 y (a + c + 2)!(r-l)!(a + 2-r)! 

X (C + 2)a+l-rib + 'S)r-2{C + l)a+2(c + 3)a(6 + 3 - s)^ 



a+2 

X 

fc=4 



Y[{{b + c + k)-+^-'') {{b + 2){a + 1) - (r - 1)(6 + 2 - s)) 



Proof. Using the argumentation preceding equations ^ and we get 

det M^{a,b,c,0,s,t) = det M^(a, 6, c, 1, s, i) (11) 
detM^{a,b,c,a + 3,s,t) = det M^{a,b, c,a + 2, s,t). (12) 

Now we can prove the claimed expression for det Mq by induction on a. It is easily checked that the 
statement of Lemma ^ holds for a = 1 . Equation ^) with A = gives 

(detM°)(detMo";^;+^) = (det M°;,i)(det M°;+^) - (det Af°f +i)(det m",^,). (13) 



We will express the occurring determinants in terms of the determinants of Mq and Mq'^ to be able 
to carry out the induction. We do this by relabelling rows and columns and get: 

det M°;l::+l{a, 6, c, r, *, *) = det Mo%"(a - 1, 5, c, r ^ 1, *, *), (14) 

det MQ''l{a, b, c, r, s, *) = det Afg (a — 1, 6, c, r — 1, s, *), (15) 

det Mp'^+i (a, 6, c,r,s,*) = det M^(a - 1,6- l,c+ l,r,s - 1,*), (16) 

det M°;f +i(a, b, c, r, *, *) = det M°;„"+ii(a, 6 + 1, c - 1, r - 1, *, *). (17) 

The matrices of the form Mg occurring in the above equations (^^-(^7|) have parameter (a — 1) instead 
of a, so we can carry out the induction step by using the values for det Afg derived in Lemma ^ and 
the induction hypothesis for det A/q. If r > 2, cancellation of common factors in the two sides of ( p^ ) 
yields the identity 

((6 + 2)(a+l) - (r- l)(6 + 2-s))(a + 5 + 2-s) 

= ((6 + 2)a - (r - 2)(6 + 2 - s)) (a + 6 + 2) - ((6 + l)a - (r - 1)(6 + 2 - s))s, 
which is easily seen to be valid. The case r — 1 can be done analogously using equations (^) and 

Q. □ 



Proof of Theorem Now we know all terms of equation (Bh and can evaluate det A/. It is indeed the 
expression of Theorem ^ For, by plugging into equation ^) the claimed formula for det M and the 
expressions derived in equations (|^)~(|To|) and in Lemma |^, we get an equation that can be simplified 
by cancelling common factors. Ifl<r<a+1 the remaining identity is 

[b + l)(c + 1)((& + 2)(a + 1) - (r - 1)(6 + 2 - s)) ((c + 2)(a + 1) - (a + 1 - r)t) 

- s(c + 2 - t){{a + l)(c + 1) - (a + 2 - r)t) {{a + \){b + 1) - r(6 + 2 - s)) 
= (a + 1)(6 + l)(c + l)(a + 2 - r)(6 + 2 - .s)(c + 2 - t) + (a + 1)(6 + l)(c + l)rst 

- (a + 2 - r)(5 + 2 - s)(c + 2 - t)rst + (a + l)(c + 1)(6 + 2 - s)(c + 2 - t)rs 

+ {b+ l)(a + l)(c + 2 - t)(a + 2 - r)st + {c + l){h + l){a + 2 - r){b + 2 - s)tr, 

which is easily verified. The cases r = and r — a -\- 2 can be done analogously using equations (^, 
(0), (HI) and (|l|). Thus the proof of Theorem |l| is complete. □ 
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